Retrofitting Distributional Embeddings to Knowledge Graphs with Functional Relations

Methods for retrofitting representations learned from distributional data to the structure of a knowledge graph typically assume that entities are embedded in a connected space and that relations imply similarity. However, knowledge graphs containing diverse entities and relations often do not accord with these assumptions. To overcome these limitations, we present a framework that generalizes existing retrofitting methods by explicitly modelling pairwise relations. We show that a simple instantiation of this framework with linear relational functions significantly outperforms existing retrofitting methods on complex knowledge graphs and loses no accuracy on graphs where relations do imply similarity. Finally, we demonstrate the utility of this method by predicting new drug–disease treatment pairs in a large, complex health knowledge graph. Introduction Distributional representations of concepts are often easy to obtain from unstructured data sets, but they tend to provide only a blurry picture of the relationships that exist between concepts. In contrast, knowledge graphs directly encode this relational information, but it can be difficult to summarize the graph structure in a single representation for each entity. To combine the advantages of distributional and relational data, Faruqui et al. (2014) propose to retrofit embeddings learned from distributional data to the structure of a knowledge graph. Their method first learns entity representations based solely on distributional data and then applies a retrofitting step to update the representations based on the structure of a knowledge graph. This modular approach conveniently separates the distributional data and entity representation learning from the knowledge graph and retrofitting model, allowing one to flexibly combine, reuse, and adapt existing representations to new tasks. However, a core assumption of Faruqui et al.’s retrofitting model is that connected entities should have similar embeddings. This assumption often fails to hold in large, complex knowledge graphs, for a variety of reasons. First, subgraphs of a knowledge graph often contain distinct classes of entities that are most naturally embedded in disconnected vector spaces. In the extreme case, the representations for these entities might derive from very different underlying data sets. Athelas Kingsfoil